1. Field of the Invention
The present invention relates to an information processing apparatus, a three-dimensional position calculation method, and a program.
2. Description of the Related Art
In recent years, researches on the mixed reality (MR) technology have been actively undertaken. The MR technology is a technology for seamlessly mixing the virtual space generated by a computer with the real space. The MR technology is expected to be applied to diverse fields, for example, assembly support operations where assembly procedures are displayed with wiring conditions superimposed thereon, and surgery support operations where a patient's body surface is displayed with body's internal conditions superimposed thereon.
To allow a user to feel that a virtual object really exists in the real space, it is essential to achieve geometric consistency between the virtual object and the real space. There are two different geometric consistency schemes in the mixed reality: a consistency scheme for conforming the coordinate system of the real space to the coordinate system of the virtual space, and a consistency scheme for correctly expressing an anteroposterior relation between real and virtual objects. A problem dealing with the former consistency scheme is also referred to as a registration problem in the mixed reality. Various researches on the registration problem are currently being conducted. A problem dealing with the latter consistency scheme is also referred to as an occlusion problem. In particular, the occlusion problem is crucial also for a video see-through type MR system which superimposes a virtual object on an image shot by a camera. The present application deals with the latter consistency, i.e., the occlusion problem.
To correctly express an anteroposterior relation between real and virtual objects, i.e., hiding (occlusion), it is necessary to obtain three-dimensional position information for the real or virtual object to be hidden. Specifically, an information processing apparatus compares three-dimensional position information for the real object with three-dimensional position information for the virtual object and, if the real object is anterior to the virtual object, displays the shot real object's image on the anterior side. If the virtual object is anterior to the real object, the information processing apparatus can display the virtual object on the anterior side. In this processing, since there is a known three-dimensional model for the virtual object, the information processing apparatus can calculate a three-dimensional position of the virtual object with respect to the viewpoint. However, since the three-dimensional position of the real object with respect to the viewpoint remains unknown only by shooting the real object, the information processing apparatus needs to obtain the three-dimensional position of the real object.
A technique for measuring a three-dimensional position of a real object will be described below. A general three-dimensional position measurement technique applies matching processing to a point of interest in images shot by a stereo camera, based on epipolar constraint and pixel patch luminance information. Specifically, with respect to a point of interest in one image shot by the stereo camera, the information processing apparatus recognizes as a corresponding point candidate a point on an epipolar line in the other image shot thereby. Then, the information processing apparatus performs pattern matching between pixel patches around each point remaining as a corresponding point candidate and pixel patches around a point of interest to obtain corresponding points with sufficient accuracy. (Hideyuki Tamura, “Computer Image Processing”, Ohmsha, Ltd., 2002, and Jun Sato, “Computer Vision—Geometry of Vision”, Corona Publishing Co., Ltd., 1999).
The conventional stereo matching, a general three-dimensional position measurement technique, cannot perform real-time processing in some cases because of a large amount of calculations in correspondence processing through pattern matching based on pixel value correlations.